feat(expr): add bound expressions - #9125
Conversation
Codecov Report❌ Patch coverage is
☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
Merging this PR will not alter performance
Comparing Footnotes
|
|
There is a performance regression, but we would get a win if we computed return_dtype for a bound expr! But binding may be slow (once). |
Polar Signals Profiling ResultsLatest Run
Powered by Polar Signals Cloud |
Benchmarks: String Encoding 📖vortex / vortex-file-compressed / ms (1.004x ➖, 0↑ 0↓)
vortex / vortex-file-compressed / % (1.000x ➖, 0↑ 0↓)
|
Benchmarks: PolarSignals Profiling 📖Vortex (geomean): 1.011x ➖ datafusion / vortex-file-compressed / ns (1.011x ➖, 0↑ 1↓)
No file size changes detected. |
Benchmarks: FineWeb NVMe 📖Verdict: No clear signal (low confidence) How to read Verdict and Engines
datafusion / vortex-file-compressed / ns (0.966x ➖, 0↑ 0↓)
datafusion / parquet / ns (0.980x ➖, 0↑ 0↓)
duckdb / vortex-file-compressed / ns (0.834x ✅, 6↑ 0↓)
duckdb / parquet / ns (0.943x ➖, 1↑ 0↓)
File Size Changes (2 files changed, -46.3% overall, 0↑ 2↓)
Totals:
|
Benchmarks: TPC-H SF=1 on NVME 📖Verdict: No clear signal (low confidence) How to read Verdict and Engines
datafusion / vortex-file-compressed / ns (1.037x ➖, 0↑ 0↓)
datafusion / parquet / ns (0.992x ➖, 2↑ 1↓)
duckdb / vortex-file-compressed / ns (1.004x ➖, 1↑ 2↓)
duckdb / parquet / ns (0.995x ➖, 0↑ 0↓)
File Size Changes (9 files changed, -43.9% overall, 0↑ 9↓)
Totals:
|
Benchmarks: TPC-DS SF=1 on NVME 📖Verdict: No clear signal (low confidence) How to read Verdict and Engines
datafusion / vortex-file-compressed / ns (1.027x ➖, 0↑ 4↓)
datafusion / parquet / ns (1.004x ➖, 0↑ 0↓)
duckdb / vortex-file-compressed / ns (0.975x ➖, 11↑ 5↓)
duckdb / parquet / ns (0.997x ➖, 6↑ 7↓)
File Size Changes (25 files changed, -43.5% overall, 0↑ 25↓)
Totals:
|
Benchmarks: Random Access 📖Vortex (geomean): 1.007x ➖ vortex / vortex-file-compressed / ns (1.007x ➖, 0↑ 0↓)
vortex / parquet / ns (1.001x ➖, 0↑ 0↓)
vortex / lance / ns (1.003x ➖, 0↑ 0↓)
|
Benchmarks: Clickbench Sorted on NVME 📖Verdict: No clear signal (environment too noisy confidence) How to read Verdict and Engines
datafusion / vortex-file-compressed / ns (0.992x ➖, 1↑ 0↓)
datafusion / parquet / ns (0.936x ➖, 2↑ 0↓)
duckdb / vortex-file-compressed / ns (1.046x ➖, 0↑ 4↓)
duckdb / parquet / ns (0.960x ➖, 2↑ 0↓)
File Size Changes (201 files changed, -42.7% overall, 50↑ 151↓)
Totals:
|
Benchmarks: Statistical and Population Genetics 📖Verdict: No clear signal (low confidence) How to read Verdict and Engines
duckdb / vortex-file-compressed / ns (1.087x ➖, 0↑ 3↓)
duckdb / parquet / ns (0.993x ➖, 0↑ 0↓)
File Size Changes (2 files changed, -32.3% overall, 0↑ 2↓)
Totals:
|
Benchmarks: FineWeb S3 📖Verdict: No clear signal (environment too noisy confidence) How to read Verdict and Engines
datafusion / vortex-file-compressed / ns (1.059x ➖, 0↑ 3↓)
datafusion / parquet / ns (0.913x ➖, 0↑ 0↓)
duckdb / vortex-file-compressed / ns (1.018x ➖, 0↑ 0↓)
duckdb / parquet / ns (0.776x ➖, 1↑ 0↓)
|
Benchmarks: Appian on NVME 📖Verdict: No clear signal (low confidence) How to read Verdict and Engines
datafusion / vortex-file-compressed / ns (1.016x ➖, 0↑ 0↓)
datafusion / parquet / ns (1.009x ➖, 0↑ 0↓)
duckdb / vortex-file-compressed / ns (0.991x ➖, 0↑ 0↓)
duckdb / parquet / ns (1.001x ➖, 0↑ 0↓)
duckdb / duckdb / ns (0.999x ➖, 0↑ 0↓)
File Size Changes (1 files changed, -0.0% overall, 0↑ 1↓)
Totals:
|
Benchmarks: TPC-H SF=10 on NVME 📖Verdict: No clear signal (low confidence) How to read Verdict and Engines
datafusion / vortex-file-compressed / ns (1.017x ➖, 0↑ 0↓)
datafusion / parquet / ns (0.994x ➖, 1↑ 0↓)
duckdb / vortex-file-compressed / ns (0.959x ➖, 1↑ 0↓)
duckdb / parquet / ns (0.993x ➖, 0↑ 0↓)
File Size Changes (9 files changed, -44.0% overall, 0↑ 9↓)
Totals:
|
Benchmarks: TPC-H SF=1 on S3 📖Verdict: No clear signal (low confidence) How to read Verdict and Engines
datafusion / vortex-file-compressed / ns (1.116x ➖, 0↑ 2↓)
datafusion / parquet / ns (1.003x ➖, 0↑ 0↓)
duckdb / vortex-file-compressed / ns (1.005x ➖, 0↑ 0↓)
duckdb / parquet / ns (1.028x ➖, 0↑ 0↓)
|
Benchmarks: Clickbench on NVME 📖Verdict: No clear signal (low confidence) How to read Verdict and Engines
datafusion / vortex-file-compressed / ns (1.009x ➖, 2↑ 3↓)
datafusion / parquet / ns (1.007x ➖, 0↑ 2↓)
duckdb / vortex-file-compressed / ns (1.001x ➖, 5↑ 5↓)
duckdb / parquet / ns (0.991x ➖, 0↑ 0↓)
File Size Changes (101 files changed, -39.2% overall, 0↑ 101↓)
Totals:
|
Benchmarks: TPC-H SF=10 on S3 📖Verdict: No clear signal (environment too noisy confidence) How to read Verdict and Engines
datafusion / vortex-file-compressed / ns (1.057x ➖, 0↑ 1↓)
datafusion / vortex-compact / ns (1.003x ➖, 0↑ 0↓)
datafusion / parquet / ns (1.061x ➖, 0↑ 2↓)
duckdb / vortex-file-compressed / ns (1.018x ➖, 0↑ 0↓)
duckdb / vortex-compact / ns (1.040x ➖, 0↑ 1↓)
duckdb / parquet / ns (1.024x ➖, 0↑ 0↓)
|
Benchmarks: Vortex queries 📖Verdict: No clear signal (environment too noisy confidence) How to read Verdict and Engines
datafusion / vortex-file-compressed / ns (1.405x ❌, 0↑ 3↓)
datafusion / parquet / ns (1.271x ❌, 0↑ 1↓)
duckdb / vortex-file-compressed / ns (1.231x ❌, 0↑ 3↓)
duckdb / parquet / ns (1.134x ❌, 0↑ 1↓)
File Size Changes (1 files changed, +0.1% overall, 1↑ 0↓)
Totals:
|
Benchmarks: Compression 📖vortex / vortex-file-compressed / ns (0.989x ➖, 0↑ 0↓)
vortex / vortex-file-compressed / bytes (1.000x ➖, 0↑ 0↓)
vortex / vortex-file-compressed / ratio (0.992x ➖, 0↑ 0↓)
vortex / parquet / ns (1.001x ➖, 0↑ 0↓)
vortex / parquet / bytes (1.000x ➖, 0↑ 0↓)
|
robert3005
left a comment
There was a problem hiding this comment.
I think this makes sense, would be nice to not have a regression but that's hard
|
I am not sure it's a regression. I guess we could update the benchmark? |
|
I think you’re right that this benchmark is not representative |
63ca608 to
621f735
Compare
Introduce a bound expr #9120
This will bind a DType to an expression.
There is a performance regression, but we would get a win if we computed return_dtype for a bound expr! But binding may be slow (once).